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INTRODUCTION: 


The  intent  of  this  investigation  was  to  automate  clinical  practice  guidelines 
(aCPGs)  in  AHLTA,  the  military  health  system’s  (MHS)  electronic  health  record 
(EHR).  The  purpose  of  the  study  was  to  examine  critical  usability  issues  in 
AHLTA,  redesign  functions  to  improve  their  usability  and  test  these  functions, 
and  then  design  and  evaluate  an  aCPG  for  asthma  patients  by  incorporating  the 
improvements  in  usability.  Four  aims  guided  the  work:  (1)  Determine  critical 
AHLTA  usability  issues  and  redesign  these  AHLTA  functions  based  upon  user- 
centered  design  principles;  (2)  In  a  laboratory  setting  using  mock  patient 
scenarios,  compare  clinicians’  human  performance  in  two  computerized 
applications — the  current  version  of  AHLTA  and  the  redesigned  functions  in  an 
AHLTA  prototype  derived  from  aim  1 ;  (3)  Identify  issues  with  aCPG  use  and 
convert  the  existing  Veteran’s  Administration  and  Department  of  Defense 
(VA/DoD)  asthma  clinical  practice  guideline  into  an  aCPG  in  AHLTA  using  user- 
centered  design  principles;  and  (4)  In  a  laboratory  setting  using  mock  patient 
scenarios,  compare  clinicians’  performance  using  the  current  asthma  aCPG 
alternate  input  mode  (AIM)  form  in  AHLTA  with  the  newly  designed  aCPG  in  an 
AHLTA  prototype  derived  from  aims  1  and  3. 

Despite  the  four  aims,  only  aim  1  was  accomplished.  As  delineated  in  the  body 
of  this  report,  the  investigators  confronted  numerous  challenges  since  receiving 
funding  in  October  2006.  Many  of  these  challenges  related  to  attempting  to 
conduct  a  research  study  that  was  heavily  dependent  on  the  AHLTA 
development  and  operational  environments.  In  fact,  operational  issues  were  the 
catalyst  for  the  decision  to  cease  efforts  on  the  original  protocol  and  to  begin 
efforts  on  a  knowledge  management  repository  and  clinical  decision  support 
protocol  under  the  leadership  of  a  different  Principal  Investigator. 

Numerous  conversations  were  held  with  key  staff  members  at  high  organization 
levels,  (e.g.,  MHS  and  CITPO).  These  conversations  related  to  the  wide  array  of 
creative  efforts  that  were  considered  in  trying  to  find  a  feasible  solution  to  keep 
the  project  viable.  Staff  from  The  Geneva  Foundation  (TGF),  Dr.  Stanley  Saiki, 
Director  of  the  Pacific  Telehealth  &  Technology  Hui,  and  the  Principal 
Investigator  communicated  formally  to  discuss  how  to  best  meet  the 
congressional  intent  of  the  award.  A  letter  detailing  various  courses  of  action 
was  sent  from  TGF  to  Ms.  Lisa  Wells-Roark  at  USAMRAA  in  October  2007. 
Because  of  issues  associated  with  AHLTA  related  to  contracting,  scheduling,  and 
deployment,  it  became  clear  in  March  2008  that  the  only  responsible  course  of 
action  was  to  cease  efforts  on  the  original  protocol  effective  30  June  2008  and  to 
begin  work  on  a  new  protocol. 

BODY: 

Task  1 :  Complete  preparatory  work  prior  to  the  grant  initiation 
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a.  Tripler  Army  Medical  Center  (TAMC)  Scientific  Review  Committee 
(SRC):  approved  the  protocol  with  modifications  on  7  November  2006. 

b.  Institutional  Review  Board  (IRB)  approvals 

1)  TAMC 

a)  Initial  approval  was  granted  on  18  December  2006  with 
stipulations. 

b)  The  start  letter  was  received  on  13  March  2007  following 
completion  of  second  level  review. 

c)  The  first  modification  was  approved  on  29  May  2007.  In  it, 
asthma  was  specified  as  the  aCPG  to  be  studied,  naturalistic 
observation  was  added  as  a  data  collection  technique  for 
Aim  1  and  a  request  was  made  to  have  all  audio-taped 
qualitative  data  transcribed.  A  new  onsite  PI  was  also  noted. 

d)  The  second  modification  was  approved  on  23  July  2007. 

Two  changes  were  listed  in  this  modification:  (a)  adding  in- 
depth  interviews  to  guide  the  development  of  the  asthma 
aCPG  and  (b)  recruiting  clinician  volunteers  from  existing 
administrative  meetings  to  validate  the  usability  findings 
instead  of  convening  focus  groups. 

e)  The  third  modification  was  approved  on  26  September  2007. 
It  was  submitted  to  allow  access  to  actual  patient  records  in 
AHLTA  during  the  interviews  that  would  be  conducted 
related  to  aCPG  development. 

f)  The  multiple  modifications,  within  the  six  months  following 
receipt  of  the  start  letter,  reflect  the  considerable  challenge 
of  attempting  to  conduct  a  research  study  that  was  so  tied  to 
the  operational  environment.  The  investigators  could  not 
anticipate  the  continuation  or  even  escalation  of  these 
challenges,  especially  as  new  versions  of  AHLTA  were 
released  at  the  centrally  managed  enterprise  level  from  the 
Clinical  Integration  Technology  Program  Office  (CITPO) 
without  consideration  for  existing  issues  and/or  studies  at 
local  (TAMC  and  other)  levels.  Evidence  supporting  this 
belief  was  brought  to  the  attention  of  the  study  team  on  23 
August  2007  when  the  PI  arranged  a  virtual  demonstration  of 
an  AHLTA  release  that  was  in  development  and  scheduled 
for  deployment  (AHLTA  3.5).  Information  gathered  in  this 
session  raised  many  questions  in  the  minds  of  the  study 
team  regarding  the  ultimate  scientific  and  operational 
contributions  of  the  project  because  the  protocol  and 
proposal  were  based  on  the  current  version  of  AHLTA.  The 
planned  release  had  already  addressed  many  of  the  usability 
issues  identified  during  the  initial  data  collection,  rendering 
that  effort  more  obsolete  than  insightful. 

2)  University  of  Utah — initial  approval  was  granted  on  29  Mar  07 
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3)  Second  level  review — the  investigators  were  directed  to  use 
Tricare  Management  Activity  (TMA)  for  the  second  level  review 
rather  than  MRMC.  TMA  approved  the  study  on  15  FEB  07. 

c.  A  subaward  was  made  to  Pacific  Science  &  Engineering  (PSE)  on  10 
December  2006. 

d.  TAMC  IMD  Proprietary  Evaluation  Group  (PEG)  approved  the 
Cooperative  Research  and  Development  Agreement.  Signature  was 
obtained  in  January  2007. 

e.  Space  was  allocated  for  the  research  team  in  D  wing  of  TAMC. 
However  in  mid-December  2007,  the  Project  Manager  (PM)  found 
mold  growing  in  both  rooms  dedicated  for  project  use;  she  also  reacted 
to  it  (e.g.,  watery  eyes).  Preventive  Medicine  was  notified  about  the 
problem.  Study  files  were  relocated  to  Nursing  Research  Service  at 
TAMC,  equipment  was  moved  to  clinical  informatics,  and  the  PM  was 
given  an  office  in  the  clinical  informatics  area.  The  rooms  in  D-wing 
were  ultimately  cleaned,  ruined  furniture  was  removed  and  the 
carpeting  was  replaced  with  tile.  On  30  June  2008  the  investigators 
assessed  any  remaining  items  in  the  rooms.  Mr.  Dale  York  of  clinical 
informatics  at  TAMC  is  aware  that  some  supplies  are  still  there  and 
these  can  be  moved  to  clinical  informatics  for  use  by  the  staff. 

f.  Source  code  for  AHLTA  computer  training  system  (CTS)  was  obtained 
from  CITPO  on  26  January  2007.  This  source  code  was  requested  by 
Pacific  Science  and  Engineering  (PSE)  to  support  and  expedite  their 
development  of  the  revised  AHLTA  prototype  interface.  However,  the 
CTS  could  not  be  used  for  protocol  analysis  because  the  patients  in 
the  CTS  were  not  sufficiently  complex  nor  was  the  CTS  maintained  at 
a  level  currently  representative  of  the  production  environment.  In 
addition,  discussions  with  CITPO  staff  and  the  CTS  developers 
confirmed  that  the  CTS  could  not  be  modified  and  that  updates  to  CTS 
were  only  completed  with  major  AHLTA  releases. 

g.  Morae  software  was  recommended  by  PSE  to  capture  the  keystrokes 
of  AHLTA  users  during  the  tests  of  aims  1  and  3.  The  software  was 
thoroughly  tested  by  PSE  and  it  was  purchased  for  use  in  this  study. 
Subsequently,  a  variety  of  problems  related  to  Morae  surfaced, 
including  that  it  was  not  compatible  with  AHLTA.  PSE  staff  obtained  a 
beta  version  of  Morae  that  was  compatible  with  AHLTA,  but  it 
adversely  impacted  network  performance.  Consequently,  PSE  self- 
developed  data  collection  software  to  capture  audio,  video  and 
observer  notes  that  were  time-synchronized  to  one  another. 

h.  Statement  of  Work  approved  by  Clinical  Investigation  Regulatory  Office 
(CIRO)  7  March  2007. 

Task  2:  Complete  grant  start-up 

a.  Project  director  hiring 

1)  The  position  was  filled  on  4  December  2006  and  vacated  on  13 
April  2007  due  to  an  unanticipated  family  issue. 
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2)  A  second  project  director  was  hired  on  1  Apr  2006.  She  submitted 
a  letter  of  resignation  effective  29  June  2007.  The  job  description 
was  re-crafted  by  the  PI  and  co-PIs  based  on  feedback  from  the 
second  project  director  and  in  consideration  of  the  needs  of  the 
study.  A  project  manager  rather  than  a  project  director  was 
recruited. 

3)  A  project  manager  (PM)  was  hired  with  a  start  date  of  27  August 
2007.  A  co-PI  made  a  site  visit  5-8  September  2007  to  orient  the 
project  manager.  Toward  the  end  of  2007  the  Pis  determined  that 
an  alternate  approach  and  timeline  for  the  study  would  be  required. 
These  changes  to  the  project  made  it  unnecessary  to  have  a  50% 
effort  PM.  The  PM  was  therefore  given  the  option  to  remain  with 
the  project  but  with  a  change  in  work  hours  to  an  as  needed  basis. 
This  did  not  suit  the  PM’s  needs,  so  she  resigned  effective  28 
January  2008. 

4)  Given  the  need  to  retool  the  study  or  close  it,  no  efforts  were  made 
to  fill  the  PM  position. 

b.  Project  plan 

1 )  A  high-level  project  plan  was  developed  during  the  second  quarter 
of  the  study  (Jan-Mar  2007). 

2)  The  project  plan  was  revised  to  adjust  for  the  protocol  modifications 
as  well  as  the  need  to  de-identify  June  data  prior  to  proceeding  with 
further  data  collection 

c.  Kick-off  meeting  held  at  TAMC  1 8  June  2007 

1)  Participants  included  the  PI,  co-PIs,  local  PI,  project  director  and 
staff  from  PSE 

2)  Topics  covered  were: 

a)  Project  history 

b)  TAMC  culture  and  complexity 

c)  Relationships  among  TATRC,  the  Hui,  and  The  Geneva 
Foundation 

d)  Discussion  of  AMEDD  Centers  of  Excellence  including  the 
current  status  of  the  concept  and  their  implementation 

e)  Impact  of  Defense  Business  Transformation  policies  on  the 
project 

f)  Project  priorities  for  this  site  visit 

3)  Additionally,  the  PI,  coPIs  and  lead  staff  member  from  PSE  met  to 
review  the  study  methodology  and  earmark  potential  dates  for 
future  work  onsite  (e.g.,  aCPG  interviews  in  September  2007, 
piloting  the  AHLTA  prototype  in  December  2007) 

d.  Advertise  the  study 

1)  The  physician  consultant,  in  collaboration  with  the  TAMC  Deputy 
Commander  for  Clinical  Services  (DCCS),  distributed  an  email 
explaining  the  study  and  encouraging  clinician  participation 

2)  A  study  brochure  was  designed,  printed,  and  distributed  at  key 
meetings  attended  by  TAMC  providers 


7 


3)  The  physician  consultant  and  project  director  attended 
departmental  administrative  meetings  to  explain  the  study 

e.  Recruit  participants  for  the  usability  data  collection  in  June  2007 

1)  The  project  director  recruited  7  clinicians  for  the  usability  interviews 

2)  The  PI  recruited  an  additional  6  clinicians  for  the  usability  interviews 

3)  The  PI  also  recruited  6  clinicians  to  participate  in  the  naturalistic 
observations 

4)  Ultimately,  12  of  the  13  clinicians  were  able  to  participate  in  the 
interviews.  One  of  the  clinicians  who  participated  in  the  naturalistic 
observations  was  also  interviewed  yielding  a  total  of  17  participants 
in  data  collection  related  to  study  aim  1 . 

Task  3:  Construct  initial  Functional  Decomposition  Diagrams  (FDDs) 

a.  A  literature  review  was  completed  but  no  FDDs  were  located  relevant 
to  the  study  environment. 

b.  The  FDDs  were  to  be  developed  by  PSE  from  the  June  2007  data 
During  the  analysis  of  the  June  2007  data,  PSE  determined  that  the 
information  required  to  prepare  the  FDDs  was  not  available  in  the  data. 
In  place  of  the  FDDs,  PSE  developed  rudimentary  workflow  diagrams 
which  were  provided  to  the  Pi’s  in  their  analysis  report. 

Task  4:  Interview,  audiotape,  and/or  videotape  participants  about  AHLTA 
usability 

a.  A  co-PI  and  the  PSE  staff  completed  a  site  visit  to  TAMC  in  May  2007 
to  ensure  site  readiness  for  the  June  data  collection 

1)  Network  connections  were  installed  in  the  designated  office  space 

2)  Power  was  restored  to  a  wall  in  one  of  the  offices 

3)  All  equipment  was  tested  and  problems  were  solved  (e.g., 
cameras,  computers) 

4)  Other  equipment  needed  for  June  data  collection  was  acquired 

5)  The  interview  guide  was  developed  in  conjunction  with  PSE 

b.  Meet  with  individual  clinicians  -  completed  in  June  2007 

1)  Twelve  individual  clinician  interviews  were  completed;  one  provider 
was  ill  and  could  not  be  rescheduled 

2)  Six  naturalistic  observations  were  completed 

3)  During  June  2007,  sufficient  data  were  collected  to  conduct  the 
analysis  to  meet  Aim  1 

c.  Analyze  audio  and  videotapes 

1 )  During  June  data  collection,  some  clinician  participants  chose  to 
use  APILTA  screens  from  their  panels  of  patients  as  they  were 
better  able  to  illustrate  usability  issues.  This  was  not  intended.  The 
plan  in  the  protocol  was  to  have  providers  use  patients  in  the 
AHLTA  CTS  (computer  training  system)  to  demonstrate  their 
concerns  with  AHLTA.  Because  actual  patient  records  were  used 
during  data  collection,  the  investigators  knew  these  data  would 
need  to  be  de-identified.  They  also  brought  this  unintended 
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occurrence  to  the  attention  of  the  Human  Use  Committee  (HUC)  at 
TAMC.  The  HUC  was  grateful  for  the  investigators’  forthrightness 
and  found  the  plans  to  de-identify  the  June  data  adequate  to 
protect  information  from  actual  patients. 

2)  The  analysis  of  the  June  data  was  put  on  hold  until  the  patient  data 
were  de-identified  or  redacted  by  PSE.  This  proved  to  be  a  rather 
lengthy  and  complex  process.  Documents  had  to  be  staffed 
through  numerous  individuals  for  approval,  including  USAMRAA. 
The  data  were  finally  redacted  and  analyzed  during  the  first  quarter 
of  2008  (Oct-Dec  2007). 

3)  PSE  delivered  their  analysis  in  a  report  dated  January  2008.  The 
Pis  had  a  number  of  concerns  about  the  report.  Among  these  was 
the  request  to  provide  the  FDDs  which  had  been  promised  as  a 
part  of  the  analysis.  It  was  at  this  point  the  investigators  learned 
the  information  required  to  prepare  the  FDDs  was  not  available  in 
the  June  data.  A  particular  disappointment  to  the  investigators  was 
the  absence  of  high  resolution  screen  shots  as  they  are  important 
to  complete  manuscripts.  The  investigators  learned  that  the  poor 
quality  of  the  screen  shots  as  due  to  having  the  June  data  de- 
identified. 

4)  Although  there  were  plans  to  submit  a  manuscript  to  a  high  tier 
research  journal,  this  is  not  possible  the  Pi’s  plan  to  prepare  a 
paper  to  submit  for  publication  based  on  the  usability  analysis  that 
was  done.  Because  of  the  nature  of  the  analysis,  the  manuscript 
will  focus  on  methods  to  remedy  usability  issues  in  AHLTA. 

5)  As  an  alternative,  the  investigators  are  working  on  a  manuscript  to 
be  submitted  to  Military  Medicine. 

6)  The  work  has  not  progressed  beyond  this  point. 

d.  Collate  usability  issues  with  the  current  version  of  AHLTA:  this  work 
was  not  done  because  of  issues  related  to  the  inability  to  get  access  to 
future  releases  of  AHLTA  while  they  were  in  development.  As  stated 
in  the  introduction,  AHLTA  development,  deployment,  and  operational 
issues  were  a  serious  interference  in  conducting  this  study.  The 
operational  stumbling  blocks  were  viewed  as  insurmountable  by  a 
number  of  individuals,  leading  to  the  decision  to  close  the  study  and 
return  unused  funds. 

e.  Expand  FDDs:  this  work  was  not  done. 

Task  5:  Develop  the  new  AHLTA  prototypes  with  user-centered  design 
principles — work  was  not  done  on  task  5. 

Task  6:  Complete  pilot  testing — work  was  not  done  on  task  6. 

Task  7:  Complete  the  human  performance  study  Aim  2  comparing  the 
current  version  of  AHLTA  and  the  new  AHLTA  prototype — work  was  not 
done  on  task  7. 
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Task  8:  Data  analysis  and  manuscript  writing 


a.  The  only  analysis  that  was  completed  pertained  to  the  interviews  and 
naturalistic  observations  conducted  for  aim  1 . 

b.  There  are  two  manuscripts  in  progress:  one  related  to  the  usability 
data  as  noted  in  Task  4  and  one  to  delineate  issues  related  to 
establishing  Centers  of  Excellence  at  the  Army  Regional  Medical 
Commands 

Task  9:  Identify  issues  and  requirements  for  aCPGs 

a.  The  research  team  realized  that  the  actual  aCPG  work  could  begin 
sooner  than  reflected  in  the  linear  task  list. 

b.  On  21  July  2007,  the  PI  and  co-PIs  discussed  general  areas  and  ideas 
to  be  purposed  during  the  aCPG  interviews. 

c.  On  5  August  2007  a  draft  interview  guide  for  collecting  data  related  to 
aCPG  issues  and  requirements  was  shared  with  PSE. 

d.  During  various  team  calls  lively  discussions  took  place  regarding  the 
best  focus  of  the  aCPG  interviews. 

e.  On  31  August  2007,  PSE  developed  a  separate  draft  interview  guide. 

f.  Due  to  the  insights  gained  during  a  virtual  demonstration  of 
functionality  in  future  releases  of  AHLTA,  the  investigators  did  not  do 
additional  work  on  Task  9. 

•  The  PI  explored  numerous  alternatives  for  addressing  the  remaining 
study  objectives  in  collaboration  with  Dr.  Saki  and  The  Geneva 
Foundation.  After  exploring  options  to  keep  the  original  protocol  active, 
it  was  determined  that  the  best  course  of  action  to  mitigate  any 
continued  expenditure  of  resources  was  to  work  towards  a  new  set  of 
deliverables.  As  such,  The  Geneva  Foundation  formally  closed  out  the 
research  activities  at  Tripler  Army  Medical  Center  at  the  University  of 
Utah.  No  further  scientific  efforts  were  made  on  the  award  during  this 
reporting  period. 

•  This  decision  was  communicated  to  TAMC  personnel  such  as  the 
onsite  PI  and  Department  of  Clinical  Investigations.  Appropriate 
paperwork  to  close  the  study  at  TAMC  has  been  completed. 

•  The  three  Principal  Investigators  closed  the  Tripler  Army  Medical 
Center  research  site  during  the  last  week  of  June  2008.  The  Pi’s  met 
with  command  leadership,  submitted  the  Institutional  Review  Board 
Final  Report,  cleared  out  the  research  office  and  worked  on  an  article 
for  publication.  The  three  original  investigators  are  no  longer 
supporting  current  grant  activities. 
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Work  was  not  accomplished  on  T asks  10-14. 

Task  10:  Develop  the  new  aCPGs  in  AHLTA  with  user-centered  design 

principles 

Task  11:  Complete  pilot  testing  of  the  aCPG  prototype 

Task  12:  Complete  the  human  performance  study  Aim  4  comparing  the 

current  aCPG  AIM  forms  with  the  new  aCPG  prototype  in  AHLTA 

Task  13:  Data  analysis 

Task  14:  Completing  the  final  report  and  writing  manuscripts 

KEY  RESEARCH  ACCOMPLISHMENTS: 

•  Completed  4  required  IRB  approvals 

•  Completed  data  collection  about  usability  issues 

•  Two  manuscripts  are  in  progress:  one  on  usability  issues  based  upon  the 
grant  data  collection  and  the  other  on  Centers  of  Excellence 

REPORTABLE  OUTCOMES: 

•  The  investigators  developed  a  strong  appreciation  for  conducting  a 
research  study  using  an  electronic  health  record  that  is  still  undergoing 
significant  changes,  especially  because  the  oversight,  configuration 
management,  and  development  are  managed  at  a  central  MHS  program 
office  rather  than  locally 

•  Due  to  the  decision  to  close  the  study,  PSE  was  given  a  30-day  notice  and 
their  contract  was  terminated  on  20  February  2008.  On  29  April  2008  The 
Geneva  Foundation  notified  The  University  of  Utah  that  no  further 
administrative  or  scientific  activity  towards  the  Scope  of  Work  was  to  occur 
past  30  June  2008.  It  was  requested  that  The  University  take  all 
reasonable  steps  to  reduce  financial  obligation  to  the  award.  The 
University  of  Utah  research  site  closed  effective  30  June  2008. 

•  Data  files  remain  onsite  at  TAMC  and  are  ready  to  be  inspected  as 
required  by  Army  and  other  Federal  regulations. 

CONCLUSION: 

•  Based  on  the  naturalistic  observation  of  6  providers  and  interviews  with  12 
clinicians,  significant  usability  issues  were  identified  in  the  AHLTA 
software.  The  PSE  analysis  identified  175  usability  issues,  with  more  than 
20  of  these  being  highly  critical  to  user  performance  and  efficiency.  PSE 
concluded  that  a  majority  of  the  issues  are  related  to  the  design  and  layout 
of  the  data  views  that  are  presented  to  the  providers 

•  Conducting  a  site-specific  research  study  based  on  the  current  version  of 
AHLTA  ultimately  was  determined  to  be  not  reasonable  because  it  would 
not  answer  the  ‘so  what?’  question;  this  is  because  future  releases  of 
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AHLTA — which  are  managed,  developed,  and  deployed  centrally  by  the 
MHS — are  scheduled  to  contain  functionality  that  is  relevant  to  usability 
and  aCPGs 

•  Executing  any/all  Center  of  Excellence  objectives  was  not  possible.  When 
the  proposal  was  conceived,  there  was  an  intention  by  the  Army  Medical 
Department  to  work  with  CITPO  to  establish  local  service  development 
environments  at  the  Regional  Medical  Command  levels  to  support  the 
ongoing  development  of  AHLTA.  TAMC  was  designated  by  The  Army 
Surgeon  General  to  serve  as  the  Center  of  Excellence  for  the  automation 
of  DoD/VA  CPGs  for  the  AMEDD.  Due  to  changes  in  leadership, 
acquisition  rules  and  regulations,  and  new  guidance  resulting  from  the 
Defense  Business  Transformation  initiative,  there  have  been  no  MTFs 
approved  for  establishing  a  local  service  development  environment.  This 
negatively  impacted  the  ability  of  TAMC  to  serve  as  a  Center  of 
Excellence  for  automating  DoD/VA  approved  CPGs  as  well  as  for 
accomplishing  the  study  objectives. 


SCOPE  OF  WORK  MODIFICATION  AND  CHANGE  IN  RESEARCH  TEAM: 

•  On  8  April  2008  representatives  of  The  Geneva  Foundation  met  with 
Dr.  Stan  Saiki  and  a  TATRC  representative  in  Seattle,  WA  to  discuss 
the  closeout  of  LTC  Lasome’s  SOW  and  the  process  for  submitting  a 
protocol  modification  to  extensively  revise  the  existing  SOW  and  to 
identify  a  new  Principal  Investigator. 

•  On  16  July  2008,  The  Geneva  Foundation  submitted  a  request  to 
USAMRAA  to  modify  the  SOW,  budget,  program  end  date  and  appoint 
LTC  Nahn  Do  as  the  new  Principal  Investigator.  The  Foundation 
responded  to  USAMRAA’s  29  July  2008  budget  verification  request  on 
13  August  2008.  Modification  three  (3)  was  issued  on  20  August  2008 
approving  the  Foundation’s  16  July  request. 

•  Between  20  August  2008  and  31  September  2008  the  new  research 
team  met  extensively  by  teleconference  to  discuss  the  study  start-up 
activities.  Below  is  an  accounting  of  the  work  being  done  in  support  of 
the  revised  protocol: 

a.  Update/revise  work  plan.  A  project  plan  previously  developed  is  in 
the  process  of  being  updated,  based  on  recognition  that  technical 
work  group  priorities  will  need  to  be  driven  by  content  work  group 
requirements.  Thus,  the  team  has  been  working  on  defining  use 
cases  for  tasks  involving  content  submission,  review/editing, 
addition  to  the  knowledge  repository  (KR),  successive  modifications 
of  representation  based  on  analysis  of  content,  standardization, 
and  subsequent  retrieval  and  use  development. 
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b.  Identifying  Clinical  Decision  Support  materials.  Progress  has  been 
made  to  assemble  contributed  content  on  primary  diabetes 
management  and  definitions  related  to  diabetes  for  alerts, 
reminders,  and  recommendation  logic  rules  and  additional 
definitions  and  rules  about  management  of  diabetes-related 
conditions.  The  team  is  in  the  process  of  recasting  the  previously 
submitted  content  into  the  format  of  a  prototype  submission 
template,  in  order  to  elucidate  its  appropriateness  and  need  for  any 
modification.  Subsequent  tasks  will  look  at  requirements  for 
methods  for  comparing  different  rules,  abstracting  and  generalizing 
them,  and  identifying  axes  on  which  similar  rules  differ.  Order  sets 
continue  to  be  deferred  for  the  current  period. 

c.  Refining  processes  for  distributed  research  and  coordination  with 
technical  work  group.  The  project  plan  is  aimed  at  continuing  to 
identify  technical  issues  in  supporting  the  visualization,  tagging, 
representation,  retrieval,  and  secondary  adaptation  of  rules  in  the 
knowledge  base,  as  driven  by  the  work  of  the  content  work  group. 
These  issues  involve  coordination  with  the  Technical  WG.  This 
process  is  occurring  through  continued  participation  of  a  liaison 
between  the  content  and  technical  work  groups 

2.  Technical  WG 

a.  Reviewing  various  applications  and  software  for  Clinical  Decision 
Support.  This  includes  the  JBoss  platform,  JBoss  Rules  Engine  and 
the  jBPM  engine.  Various  technical  requirements  and  subscriptions 
have  been  identified  and  ordered  for. 

b.  Conducted  a  methodical  interview  process  for  candidates  for  the 
Java  Developer  position.  The  Technical  WG  has  interviewed  7-8 
candidates  and  they  have  been  ranked  in  the  order  of  fit  and 
expertise  based  on  the  development  requirements  of  the  project. 
These  candidates  will  be  further  tested  for  their  knowledge  of  Glass 
Fish,  an  Application  Server,  which  is  one  of  the  candidate  tools 
being  planned  for  use  in  the  project.  The  technical  work  group  is 
awaiting  instructions  from  the  administrative  team  to  determine  the 
next  steps  to  move  this  forward. 

c.  The  technical  work  group  will  work  on  the  use  cases  developed  by 
the  content  team  and  perform  analysis  from  a  systems  perspective, 
propose  a  technical  architecture  and  design  and  a  roadmap  for 
implementation  and  release  of  functionalities  related  to  the 
Knowledge  Management  Repository  and  clinical  guideline 
execution. 
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